Face Generation¶

In this project, you'll use generative adversarial networks to generate new images of faces.

Get the Data¶

You'll be using two datasets in this project:

MNIST
CelebA

Since the celebA dataset is complex and you're doing GANs in a project for the first time, we want you to test your neural network on MNIST before CelebA. Running the GANs on MNIST will allow you to see how well your model trains sooner.

If you're using FloydHub, set data_dir to "/input" and use the FloydHub data ID "R5KrjnANiKVhLWAkpXhNBe".

data_dir = './data'

# FloydHub - Use with data ID "R5KrjnANiKVhLWAkpXhNBe"
data_dir = '/input'


"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import helper

helper.download_extract('mnist', data_dir)
helper.download_extract('celeba', data_dir)

Downloading mnist: 9.92MB [00:01, 7.54MB/s]                            
Extracting mnist: 100%|██████████| 60.0K/60.0K [00:11<00:00, 5.41KFile/s]
Downloading celeba: 1.44GB [00:24, 60.1MB/s]

Extracting celeba...

Explore the Data¶

MNIST¶

As you're aware, the MNIST dataset contains images of handwritten digits. You can view the first number of examples by changing show_n_images.

show_n_images = 25

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
%matplotlib inline
import os
from glob import glob
from matplotlib import pyplot

mnist_images = helper.get_batch(glob(os.path.join(data_dir, 'mnist/*.jpg'))[:show_n_images], 28, 28, 'L')
pyplot.imshow(helper.images_square_grid(mnist_images, 'L'), cmap='gray')

<matplotlib.image.AxesImage at 0x7f49453530f0>

CelebA¶

The CelebFaces Attributes Dataset (CelebA) dataset contains over 200,000 celebrity images with annotations. Since you're going to be generating faces, you won't need the annotations. You can view the first number of examples by changing show_n_images.

show_n_images = 25

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
mnist_images = helper.get_batch(glob(os.path.join(data_dir, 'img_align_celeba/*.jpg'))[:show_n_images], 28, 28, 'RGB')
pyplot.imshow(helper.images_square_grid(mnist_images, 'RGB'))

<matplotlib.image.AxesImage at 0x7f493f90ab70>

Preprocess the Data¶

Since the project's main focus is on building the GANs, we'll preprocess the data for you. The values of the MNIST and CelebA dataset will be in the range of -0.5 to 0.5 of 28x28 dimensional images. The CelebA images will be cropped to remove parts of the image that don't include a face, then resized down to 28x28.

The MNIST images are black and white images with a single color channel while the CelebA images have 3 color channels (RGB color channel).

Build the Neural Network¶

You'll build the components necessary to build a GANs by implementing the following functions below:

model_inputs
discriminator
generator
model_loss
model_opt
train

Check the Version of TensorFlow and Access to GPU¶

This will check to make sure you have the correct version of TensorFlow and access to a GPU

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
from distutils.version import LooseVersion
import warnings
import tensorflow as tf

# Check TensorFlow Version
assert LooseVersion(tf.__version__) >= LooseVersion('1.0'), 'Please use TensorFlow version 1.0 or newer.  You are using {}'.format(tf.__version__)
print('TensorFlow Version: {}'.format(tf.__version__))

# Check for a GPU
if not tf.test.gpu_device_name():
    warnings.warn('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))

TensorFlow Version: 1.1.0
Default GPU Device: /gpu:0

Input¶

Implement the model_inputs function to create TF Placeholders for the Neural Network. It should create the following placeholders:

Real input images placeholder with rank 4 using image_width, image_height, and image_channels.
Z input placeholder with rank 2 using z_dim.
Learning rate placeholder with rank 0.

Return the placeholders in the following the tuple (tensor of real input images, tensor of z data)

import problem_unittests as tests

def model_inputs(image_width, image_height, image_channels, z_dim):
    """
    Create the model inputs
    :param image_width: The input image width
    :param image_height: The input image height
    :param image_channels: The number of image channels
    :param z_dim: The dimension of Z
    :return: Tuple of (tensor of real input images, tensor of z data, learning rate)
    """
    inputs_real = tf.placeholder(tf.float32, (None, image_height, image_width, image_channels), name='real_inputs')
    inputs_z = tf.placeholder(tf.float32, (None, z_dim), name='z_inputs')
    learn_rate = tf.placeholder(tf.float32, name='learning_rate')
        
    return inputs_real, inputs_z, learn_rate



"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_inputs(model_inputs)

Tests Passed

Discriminator¶

Implement discriminator to create a discriminator neural network that discriminates on images. This function should be able to reuse the variables in the neural network. Use tf.variable_scope with a scope name of "discriminator" to allow the variables to be reused. The function should return a tuple of (tensor output of the discriminator, tensor logits of the discriminator).

def discriminator(images, reuse=False):
    """
    Create the discriminator network
    :param images: Tensor of input image(s)
    :param reuse: Boolean if the weights should be reused
    :return: Tuple of (tensor output of the discriminator, tensor logits of the discriminator)
    """
    strides = 2
    with tf.variable_scope('discriminator', reuse=reuse):
        # Input layer is 28x28x3
        layer1 = tf.layers.conv2d(images, 64, 5, strides=strides, padding='same')
        relu1 = tf.maximum(0.18 * layer1, layer1)
        # 14x14x64
        
        layer2 = tf.layers.conv2d(relu1, 128, 5, strides=strides, padding='same')
        normalised2 = tf.layers.batch_normalization(layer2, training=True)
        relu2 = tf.maximum(0.18 * normalised2, normalised2)
        # 7x7x128
        
        layer3 = tf.layers.conv2d(relu2, 256, 5, strides=strides, padding='same')
        normalised3 = tf.layers.batch_normalization(layer3, training=True)
        relu3 = tf.maximum(0.18 * normalised3, normalised3)
        # 4x4x256

        # Flatten it
        flat = tf.reshape(relu3, (-1, 4*4*256))
        logits = tf.layers.dense(flat, 1)
        out = tf.sigmoid(logits)
        
        return out, logits
    
    

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_discriminator(discriminator, tf)

Tests Passed

Generator¶

Implement generator to generate an image using z. This function should be able to reuse the variables in the neural network. Use tf.variable_scope with a scope name of "generator" to allow the variables to be reused. The function should return the generated 28 x 28 x out_channel_dim images.

def generator(z, out_channel_dim, is_train=True, alpha = 0.02):
    """
    Create the generator network
    :param z: Input z
    :param out_channel_dim: The number of channels in the output image
    :param is_train: Boolean if generator is being used for training
    :return: The tensor output of the generator
    """

    with tf.variable_scope('generator', reuse = not is_train):
        layer1 = tf.layers.dense(z, 7*7*256)
        layer1 = tf.reshape(layer1, (-1, 7, 7, 256))
        layer1 = tf.layers.batch_normalization(layer1, training=is_train)
        layer1 = tf.maximum(alpha * layer1, layer1)
        
        layer2 = tf.layers.conv2d_transpose(layer1, 128, 5, strides=2, padding='same')
        layer2 = tf.layers.batch_normalization(layer2, training=is_train)
        layer2 = tf.maximum(alpha * layer2, layer2)
        
        layer3 = tf.layers.conv2d_transpose(layer2, 64, 5, strides=2, padding='same')
        layer3 = tf.layers.batch_normalization(layer3, training=is_train)
        layer3 = tf.maximum(alpha * layer3, layer3)
        
        logits = tf.layers.conv2d_transpose(layer3, out_channel_dim, 3, strides=1, padding='same')
        return tf.tanh(logits) 
        

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_generator(generator, tf)

Tests Passed

Loss¶

Implement model_loss to build the GANs for training and calculate the loss. The function should return a tuple of (discriminator loss, generator loss). Use the following functions you implemented:

discriminator(images, reuse=False)
generator(z, out_channel_dim, is_train=True)

def model_loss(input_real, input_z, out_channel_dim):
    """
    Get the loss for the discriminator and generator
    :param input_real: Images from the real dataset
    :param input_z: Z input
    :param out_channel_dim: The number of channels in the output image
    :return: A tuple of (discriminator loss, generator loss)
    """    
    # Calculate losses
    generator_model = generator(input_z, out_channel_dim, is_train=True)
    discriminator_model_real, discriminator_logits_real = discriminator(input_real, reuse=False)
    # Fake output generated by the generator model
    discriminator_model_fake, discriminator_logits_fake = discriminator(generator_model, reuse=True)

    # Discriminator loss
    discriminator_loss_real = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=discriminator_logits_real, labels = tf.ones_like(discriminator_model_real) * 0.9))
    discriminator_loss_fake = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=discriminator_logits_fake, labels = tf.zeros_like(discriminator_model_fake)))
    discriminator_loss = discriminator_loss_fake + discriminator_loss_real
    
    # Generator loss
    generator_loss = tf.reduce_mean(
        tf.nn.sigmoid_cross_entropy_with_logits(logits=discriminator_logits_fake, labels = tf.ones_like(discriminator_model_fake)))

    return discriminator_loss, generator_loss



"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_loss(model_loss)

Tests Passed

Optimization¶

Implement model_opt to create the optimization operations for the GANs. Use tf.trainable_variables to get all the trainable variables. Filter the variables with names that are in the discriminator and generator scope names. The function should return a tuple of (discriminator training operation, generator training operation).

def model_opt(d_loss, g_loss, learning_rate, beta1):
    """
    Get optimization operations
    :param d_loss: Discriminator loss Tensor
    :param g_loss: Generator loss Tensor
    :param learning_rate: Learning Rate Placeholder
    :param beta1: The exponential decay rate for the 1st moment in the optimizer
    :return: A tuple of (discriminator training operation, generator training operation)
    """
    # Get the trainable_variables, split into Generator and Discriminator parts
    t_vars = tf.trainable_variables()
    g_vars = [var for var in t_vars if var.name.startswith('generator')]
    d_vars = [var for var in t_vars if var.name.startswith('discriminator')]

    all_update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
    g_update_ops = [var for var in all_update_ops if var.name.startswith('generator')]
    d_update_ops = [var for var in all_update_ops if var.name.startswith('discriminator')]
    
    with tf.control_dependencies(d_update_ops):
        d_train_opt = tf.train.AdamOptimizer(learning_rate,beta1=beta1).minimize(d_loss, var_list=d_vars)
    with tf.control_dependencies(g_update_ops):
        g_train_opt = tf.train.AdamOptimizer(learning_rate,beta1=beta1).minimize(g_loss, var_list=g_vars)
    
    return d_train_opt, g_train_opt


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_model_opt(model_opt, tf)

Tests Passed

Neural Network Training¶

Show Output¶

Use this function to show the current output of the generator during training. It will help you determine how well the GANs is training.

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import numpy as np

def show_generator_output(sess, n_images, input_z, out_channel_dim, image_mode):
    """
    Show example output for the generator
    :param sess: TensorFlow session
    :param n_images: Number of Images to display
    :param input_z: Input Z Tensor
    :param out_channel_dim: The number of channels in the output image
    :param image_mode: The mode to use for images ("RGB" or "L")
    """
    cmap = None if image_mode == 'RGB' else 'gray'
    z_dim = input_z.get_shape().as_list()[-1]
    example_z = np.random.uniform(-1, 1, size=[n_images, z_dim])

    samples = sess.run(
        generator(input_z, out_channel_dim, False),
        feed_dict={input_z: example_z})

    images_grid = helper.images_square_grid(samples, image_mode)
    pyplot.imshow(images_grid, cmap=cmap)
    pyplot.show()

Train¶

Implement train to build and train the GANs. Use the following functions you implemented:

model_inputs(image_width, image_height, image_channels, z_dim)
model_loss(input_real, input_z, out_channel_dim)
model_opt(d_loss, g_loss, learning_rate, beta1)

Use the show_generator_output to show generator output while you train. Running show_generator_output for every batch will drastically increase training time and increase the size of the notebook. It's recommended to print the generator output every 100 batches.

def train(epoch_count, batch_size, z_dim, learning_rate, beta1, get_batches, data_shape, data_image_mode):
    """
    Train the GAN
    :param epoch_count: Number of epochs
    :param batch_size: Batch Size
    :param z_dim: Z dimension
    :param learning_rate: Learning Rate
    :param beta1: The exponential decay rate for the 1st moment in the optimizer
    :param get_batches: Function to get batches
    :param data_shape: Shape of the data
    :param data_image_mode: The image mode to use for images ("RGB" or "L")
    """    
    losses = []
    steps = 0
    n_images = 9
        
    _, image_width, image_height, image_channels = data_shape
    input_real, input_z , learning_rate_pl = model_inputs(image_width,image_height,image_channels,z_dim)
    
    d_loss, g_loss = model_loss(input_real, input_z, image_channels)
    d_opt, g_opt = model_opt(d_loss, g_loss,learning_rate_pl, beta1)
    
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        for epoch_i in range(epoch_count): 
            for batch_images in get_batches(batch_size):
                steps = steps + 1
                
                batch_z = np.random.uniform(-1, 1, size=(batch_size, z_dim))
                batch_images = batch_images * 2.0
                
                _ = sess.run(d_opt, feed_dict={input_real:batch_images,input_z:batch_z,learning_rate_pl:learning_rate})
                _ = sess.run(g_opt, feed_dict={input_real:batch_images,input_z:batch_z,learning_rate_pl:learning_rate})
                
                if steps % 5 == 0 :
                    train_loss_d = d_loss.eval({input_real:batch_images,input_z:batch_z,learning_rate_pl:learning_rate})
                    train_loss_g = g_loss.eval({input_real:batch_images,input_z:batch_z,learning_rate_pl:learning_rate})
                    print("Epoch {}/{} ___ ".format(epoch_i+1, epoch_count),
                      "Discriminator Loss: {:.4f} ___".format(train_loss_d),
                      "Generator Loss: {:.4f}".format(train_loss_g))
                
                    losses.append((train_loss_d, train_loss_g))
                    
                if steps % 50 == 0:
                    show_generator_output(sess,n_images,input_z,image_channels,data_image_mode)

MNIST¶

Test your GANs architecture on MNIST. After 2 epochs, the GANs should be able to generate images that look like handwritten digits. Make sure the loss of the generator is lower than the loss of the discriminator or close to 0.

batch_size = 16
z_dim = 128 
learning_rate = 0.0002 
beta1 = 0.4


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
epochs = 2

mnist_dataset = helper.Dataset('mnist', glob(os.path.join(data_dir, 'mnist/*.jpg')))
with tf.Graph().as_default():
    train(epochs, batch_size, z_dim, learning_rate, beta1, mnist_dataset.get_batches,
          mnist_dataset.shape, mnist_dataset.image_mode)

Epoch 1/2 ___  Discriminator Loss: 2.1114 ___ Generator Loss: 0.2357
Epoch 1/2 ___  Discriminator Loss: 1.1770 ___ Generator Loss: 0.7391
Epoch 1/2 ___  Discriminator Loss: 5.0926 ___ Generator Loss: 0.0138
Epoch 1/2 ___  Discriminator Loss: 3.1944 ___ Generator Loss: 0.0872
Epoch 1/2 ___  Discriminator Loss: 3.2800 ___ Generator Loss: 0.1547
Epoch 1/2 ___  Discriminator Loss: 3.4413 ___ Generator Loss: 0.0843
Epoch 1/2 ___  Discriminator Loss: 3.0455 ___ Generator Loss: 0.0904
Epoch 1/2 ___  Discriminator Loss: 2.3452 ___ Generator Loss: 0.2403
Epoch 1/2 ___  Discriminator Loss: 3.4094 ___ Generator Loss: 0.0810
Epoch 1/2 ___  Discriminator Loss: 1.5583 ___ Generator Loss: 1.1792

Epoch 1/2 ___  Discriminator Loss: 2.0913 ___ Generator Loss: 0.4434
Epoch 1/2 ___  Discriminator Loss: 2.3582 ___ Generator Loss: 0.1786
Epoch 1/2 ___  Discriminator Loss: 1.5382 ___ Generator Loss: 0.5663
Epoch 1/2 ___  Discriminator Loss: 1.8014 ___ Generator Loss: 0.3659
Epoch 1/2 ___  Discriminator Loss: 1.9819 ___ Generator Loss: 0.3340
Epoch 1/2 ___  Discriminator Loss: 2.5919 ___ Generator Loss: 0.1277
Epoch 1/2 ___  Discriminator Loss: 2.0752 ___ Generator Loss: 0.3912
Epoch 1/2 ___  Discriminator Loss: 1.7728 ___ Generator Loss: 0.3194
Epoch 1/2 ___  Discriminator Loss: 2.0701 ___ Generator Loss: 0.2281
Epoch 1/2 ___  Discriminator Loss: 1.5974 ___ Generator Loss: 0.5544

Epoch 1/2 ___  Discriminator Loss: 1.7104 ___ Generator Loss: 0.7513
Epoch 1/2 ___  Discriminator Loss: 1.9677 ___ Generator Loss: 0.5165
Epoch 1/2 ___  Discriminator Loss: 2.1239 ___ Generator Loss: 0.2330
Epoch 1/2 ___  Discriminator Loss: 1.9246 ___ Generator Loss: 0.2660
Epoch 1/2 ___  Discriminator Loss: 1.9095 ___ Generator Loss: 0.2986
Epoch 1/2 ___  Discriminator Loss: 1.6222 ___ Generator Loss: 0.4150
Epoch 1/2 ___  Discriminator Loss: 1.5258 ___ Generator Loss: 0.5164
Epoch 1/2 ___  Discriminator Loss: 1.6377 ___ Generator Loss: 0.4142
Epoch 1/2 ___  Discriminator Loss: 2.6664 ___ Generator Loss: 0.1241
Epoch 1/2 ___  Discriminator Loss: 1.5991 ___ Generator Loss: 0.4401

Epoch 1/2 ___  Discriminator Loss: 2.2382 ___ Generator Loss: 0.1928
Epoch 1/2 ___  Discriminator Loss: 1.6265 ___ Generator Loss: 0.4914
Epoch 1/2 ___  Discriminator Loss: 1.5186 ___ Generator Loss: 0.5889
Epoch 1/2 ___  Discriminator Loss: 1.4295 ___ Generator Loss: 0.5194
Epoch 1/2 ___  Discriminator Loss: 1.3697 ___ Generator Loss: 1.7076
Epoch 1/2 ___  Discriminator Loss: 1.3606 ___ Generator Loss: 1.0024
Epoch 1/2 ___  Discriminator Loss: 1.2288 ___ Generator Loss: 0.6911
Epoch 1/2 ___  Discriminator Loss: 1.6331 ___ Generator Loss: 0.4162
Epoch 1/2 ___  Discriminator Loss: 1.5688 ___ Generator Loss: 0.5097
Epoch 1/2 ___  Discriminator Loss: 1.9374 ___ Generator Loss: 0.2496

Epoch 1/2 ___  Discriminator Loss: 1.0880 ___ Generator Loss: 0.9769
Epoch 1/2 ___  Discriminator Loss: 1.7290 ___ Generator Loss: 0.3285
Epoch 1/2 ___  Discriminator Loss: 1.1737 ___ Generator Loss: 1.0552
Epoch 1/2 ___  Discriminator Loss: 1.4172 ___ Generator Loss: 0.5261
Epoch 1/2 ___  Discriminator Loss: 1.5490 ___ Generator Loss: 0.4258
Epoch 1/2 ___  Discriminator Loss: 1.3021 ___ Generator Loss: 0.5786
Epoch 1/2 ___  Discriminator Loss: 1.2429 ___ Generator Loss: 0.7092
Epoch 1/2 ___  Discriminator Loss: 1.2408 ___ Generator Loss: 0.7212
Epoch 1/2 ___  Discriminator Loss: 1.1197 ___ Generator Loss: 0.7963
Epoch 1/2 ___  Discriminator Loss: 1.2655 ___ Generator Loss: 0.9124

Epoch 1/2 ___  Discriminator Loss: 1.2771 ___ Generator Loss: 0.6031
Epoch 1/2 ___  Discriminator Loss: 1.2747 ___ Generator Loss: 0.8805
Epoch 1/2 ___  Discriminator Loss: 1.2936 ___ Generator Loss: 0.6056
Epoch 1/2 ___  Discriminator Loss: 1.2145 ___ Generator Loss: 0.6432
Epoch 1/2 ___  Discriminator Loss: 1.3478 ___ Generator Loss: 0.5246
Epoch 1/2 ___  Discriminator Loss: 1.1239 ___ Generator Loss: 0.6819
Epoch 1/2 ___  Discriminator Loss: 0.9678 ___ Generator Loss: 0.9714
Epoch 1/2 ___  Discriminator Loss: 0.9433 ___ Generator Loss: 1.0817
Epoch 1/2 ___  Discriminator Loss: 0.9903 ___ Generator Loss: 1.0507
Epoch 1/2 ___  Discriminator Loss: 1.1844 ___ Generator Loss: 0.6612

Epoch 1/2 ___  Discriminator Loss: 0.9953 ___ Generator Loss: 0.8497
Epoch 1/2 ___  Discriminator Loss: 1.1875 ___ Generator Loss: 0.7035
Epoch 1/2 ___  Discriminator Loss: 1.7732 ___ Generator Loss: 0.3380
Epoch 1/2 ___  Discriminator Loss: 1.4566 ___ Generator Loss: 0.5422
Epoch 1/2 ___  Discriminator Loss: 1.2125 ___ Generator Loss: 1.9829
Epoch 1/2 ___  Discriminator Loss: 1.3067 ___ Generator Loss: 0.6639
Epoch 1/2 ___  Discriminator Loss: 1.9025 ___ Generator Loss: 0.3061
Epoch 1/2 ___  Discriminator Loss: 1.0661 ___ Generator Loss: 0.8882
Epoch 1/2 ___  Discriminator Loss: 0.9476 ___ Generator Loss: 1.5206
Epoch 1/2 ___  Discriminator Loss: 0.7799 ___ Generator Loss: 1.3640

Epoch 1/2 ___  Discriminator Loss: 1.0742 ___ Generator Loss: 0.9457
Epoch 1/2 ___  Discriminator Loss: 0.9469 ___ Generator Loss: 1.0689
Epoch 1/2 ___  Discriminator Loss: 0.7811 ___ Generator Loss: 1.1632
Epoch 1/2 ___  Discriminator Loss: 0.8621 ___ Generator Loss: 1.3934
Epoch 1/2 ___  Discriminator Loss: 1.1276 ___ Generator Loss: 0.7182
Epoch 1/2 ___  Discriminator Loss: 0.7998 ___ Generator Loss: 1.3009
Epoch 1/2 ___  Discriminator Loss: 1.8311 ___ Generator Loss: 0.3204
Epoch 1/2 ___  Discriminator Loss: 1.0535 ___ Generator Loss: 0.7818
Epoch 1/2 ___  Discriminator Loss: 0.8295 ___ Generator Loss: 1.2334
Epoch 1/2 ___  Discriminator Loss: 0.9437 ___ Generator Loss: 0.9555

Epoch 1/2 ___  Discriminator Loss: 0.8861 ___ Generator Loss: 1.1427
Epoch 1/2 ___  Discriminator Loss: 0.8880 ___ Generator Loss: 1.5909
Epoch 1/2 ___  Discriminator Loss: 1.0501 ___ Generator Loss: 1.3823
Epoch 1/2 ___  Discriminator Loss: 1.2303 ___ Generator Loss: 0.6767
Epoch 1/2 ___  Discriminator Loss: 0.7937 ___ Generator Loss: 1.2989
Epoch 1/2 ___  Discriminator Loss: 0.8591 ___ Generator Loss: 1.2419
Epoch 1/2 ___  Discriminator Loss: 1.6704 ___ Generator Loss: 0.3672
Epoch 1/2 ___  Discriminator Loss: 0.9222 ___ Generator Loss: 1.2410
Epoch 1/2 ___  Discriminator Loss: 1.8449 ___ Generator Loss: 0.2731
Epoch 1/2 ___  Discriminator Loss: 0.9831 ___ Generator Loss: 1.0255

Epoch 1/2 ___  Discriminator Loss: 1.5715 ___ Generator Loss: 0.5262
Epoch 1/2 ___  Discriminator Loss: 1.3602 ___ Generator Loss: 0.6627
Epoch 1/2 ___  Discriminator Loss: 1.4628 ___ Generator Loss: 0.5690
Epoch 1/2 ___  Discriminator Loss: 1.6010 ___ Generator Loss: 0.3979
Epoch 1/2 ___  Discriminator Loss: 1.1763 ___ Generator Loss: 0.7203
Epoch 1/2 ___  Discriminator Loss: 0.6930 ___ Generator Loss: 2.0236
Epoch 1/2 ___  Discriminator Loss: 1.2130 ___ Generator Loss: 2.3688
Epoch 1/2 ___  Discriminator Loss: 1.4907 ___ Generator Loss: 0.5157
Epoch 1/2 ___  Discriminator Loss: 1.2774 ___ Generator Loss: 0.7240
Epoch 1/2 ___  Discriminator Loss: 0.9424 ___ Generator Loss: 1.0381

CelebA¶

Run your GANs on CelebA. It will take around 20 minutes on the average GPU to run one epoch. You can run the whole epoch or stop when it starts to generate realistic faces.

batch_size = 64
z_dim = 100
learning_rate = 0.0002
beta1 = 0.5


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
epochs = 4

celeba_dataset = helper.Dataset('celeba', glob(os.path.join(data_dir, 'img_align_celeba/*.jpg')))
with tf.Graph().as_default():
    train(epochs, batch_size, z_dim, learning_rate, beta1, celeba_dataset.get_batches,
          celeba_dataset.shape, celeba_dataset.image_mode)

Epoch 1/4 ___  Discriminator Loss: 1.0455 ___ Generator Loss: 0.8712
Epoch 1/4 ___  Discriminator Loss: 0.5375 ___ Generator Loss: 2.2574
Epoch 1/4 ___  Discriminator Loss: 0.5138 ___ Generator Loss: 2.3341
Epoch 1/4 ___  Discriminator Loss: 0.6076 ___ Generator Loss: 2.1231
Epoch 1/4 ___  Discriminator Loss: 0.4019 ___ Generator Loss: 3.5059
Epoch 1/4 ___  Discriminator Loss: 0.3954 ___ Generator Loss: 5.2672
Epoch 1/4 ___  Discriminator Loss: 0.4651 ___ Generator Loss: 6.7905
Epoch 1/4 ___  Discriminator Loss: 1.4609 ___ Generator Loss: 0.7168
Epoch 1/4 ___  Discriminator Loss: 1.5398 ___ Generator Loss: 0.4757
Epoch 1/4 ___  Discriminator Loss: 0.5280 ___ Generator Loss: 3.3452

Epoch 1/4 ___  Discriminator Loss: 1.3030 ___ Generator Loss: 5.4534
Epoch 1/4 ___  Discriminator Loss: 0.9831 ___ Generator Loss: 1.1048
Epoch 1/4 ___  Discriminator Loss: 0.9155 ___ Generator Loss: 1.7393
Epoch 1/4 ___  Discriminator Loss: 0.7867 ___ Generator Loss: 1.8174
Epoch 1/4 ___  Discriminator Loss: 0.8319 ___ Generator Loss: 3.2243
Epoch 1/4 ___  Discriminator Loss: 0.8488 ___ Generator Loss: 1.2451
Epoch 1/4 ___  Discriminator Loss: 0.8917 ___ Generator Loss: 1.1578
Epoch 1/4 ___  Discriminator Loss: 0.8774 ___ Generator Loss: 1.7253
Epoch 1/4 ___  Discriminator Loss: 0.7436 ___ Generator Loss: 2.3895
Epoch 1/4 ___  Discriminator Loss: 0.7001 ___ Generator Loss: 2.4485

Epoch 1/4 ___  Discriminator Loss: 0.9928 ___ Generator Loss: 1.1051
Epoch 1/4 ___  Discriminator Loss: 0.7941 ___ Generator Loss: 1.3045
Epoch 1/4 ___  Discriminator Loss: 0.8095 ___ Generator Loss: 1.9566
Epoch 1/4 ___  Discriminator Loss: 1.0916 ___ Generator Loss: 3.2099
Epoch 1/4 ___  Discriminator Loss: 0.8429 ___ Generator Loss: 1.3567
Epoch 1/4 ___  Discriminator Loss: 0.7633 ___ Generator Loss: 2.1577
Epoch 1/4 ___  Discriminator Loss: 0.8510 ___ Generator Loss: 1.4275
Epoch 1/4 ___  Discriminator Loss: 0.5373 ___ Generator Loss: 2.4951
Epoch 1/4 ___  Discriminator Loss: 0.7036 ___ Generator Loss: 1.6257
Epoch 1/4 ___  Discriminator Loss: 0.8592 ___ Generator Loss: 3.8031

Epoch 1/4 ___  Discriminator Loss: 1.2460 ___ Generator Loss: 0.7072
Epoch 1/4 ___  Discriminator Loss: 0.6961 ___ Generator Loss: 1.9753
Epoch 1/4 ___  Discriminator Loss: 0.7090 ___ Generator Loss: 1.4952
Epoch 1/4 ___  Discriminator Loss: 0.8064 ___ Generator Loss: 1.3050
Epoch 1/4 ___  Discriminator Loss: 0.5068 ___ Generator Loss: 2.9799
Epoch 1/4 ___  Discriminator Loss: 1.4159 ___ Generator Loss: 0.6694
Epoch 1/4 ___  Discriminator Loss: 0.8770 ___ Generator Loss: 1.4013
Epoch 1/4 ___  Discriminator Loss: 0.8783 ___ Generator Loss: 1.2572
Epoch 1/4 ___  Discriminator Loss: 1.0688 ___ Generator Loss: 2.5333
Epoch 1/4 ___  Discriminator Loss: 0.5815 ___ Generator Loss: 2.7072

Epoch 1/4 ___  Discriminator Loss: 0.9152 ___ Generator Loss: 1.1848
Epoch 1/4 ___  Discriminator Loss: 0.6186 ___ Generator Loss: 2.5065
Epoch 1/4 ___  Discriminator Loss: 0.9378 ___ Generator Loss: 1.7488
Epoch 1/4 ___  Discriminator Loss: 0.6124 ___ Generator Loss: 1.9840
Epoch 1/4 ___  Discriminator Loss: 0.6686 ___ Generator Loss: 2.0727
Epoch 1/4 ___  Discriminator Loss: 0.7472 ___ Generator Loss: 2.2082
Epoch 1/4 ___  Discriminator Loss: 0.7237 ___ Generator Loss: 1.4483
Epoch 1/4 ___  Discriminator Loss: 2.2632 ___ Generator Loss: 4.7526
Epoch 1/4 ___  Discriminator Loss: 1.2715 ___ Generator Loss: 0.9428
Epoch 1/4 ___  Discriminator Loss: 1.0288 ___ Generator Loss: 0.9786

Epoch 1/4 ___  Discriminator Loss: 0.9335 ___ Generator Loss: 1.4356
Epoch 1/4 ___  Discriminator Loss: 0.7111 ___ Generator Loss: 1.6079
Epoch 1/4 ___  Discriminator Loss: 0.9187 ___ Generator Loss: 2.4815
Epoch 1/4 ___  Discriminator Loss: 0.6260 ___ Generator Loss: 2.2640
Epoch 1/4 ___  Discriminator Loss: 0.6850 ___ Generator Loss: 1.5801
Epoch 1/4 ___  Discriminator Loss: 1.1159 ___ Generator Loss: 0.7531
Epoch 1/4 ___  Discriminator Loss: 0.5345 ___ Generator Loss: 2.4135
Epoch 1/4 ___  Discriminator Loss: 0.7305 ___ Generator Loss: 3.0226
Epoch 1/4 ___  Discriminator Loss: 0.7121 ___ Generator Loss: 1.5395
Epoch 1/4 ___  Discriminator Loss: 0.5607 ___ Generator Loss: 2.1658

Epoch 1/4 ___  Discriminator Loss: 0.7497 ___ Generator Loss: 1.4276
Epoch 1/4 ___  Discriminator Loss: 0.6499 ___ Generator Loss: 2.1061
Epoch 1/4 ___  Discriminator Loss: 0.6736 ___ Generator Loss: 1.5592
Epoch 1/4 ___  Discriminator Loss: 0.5842 ___ Generator Loss: 2.4665
Epoch 1/4 ___  Discriminator Loss: 1.0969 ___ Generator Loss: 0.8060
Epoch 1/4 ___  Discriminator Loss: 1.1792 ___ Generator Loss: 1.1543
Epoch 1/4 ___  Discriminator Loss: 1.3517 ___ Generator Loss: 4.3317
Epoch 1/4 ___  Discriminator Loss: 1.0319 ___ Generator Loss: 0.9530
Epoch 1/4 ___  Discriminator Loss: 0.9392 ___ Generator Loss: 1.6723
Epoch 1/4 ___  Discriminator Loss: 1.6136 ___ Generator Loss: 0.5176

Epoch 1/4 ___  Discriminator Loss: 0.6228 ___ Generator Loss: 2.3995
Epoch 1/4 ___  Discriminator Loss: 0.8296 ___ Generator Loss: 1.7419
Epoch 1/4 ___  Discriminator Loss: 0.6920 ___ Generator Loss: 1.7868
Epoch 1/4 ___  Discriminator Loss: 0.9675 ___ Generator Loss: 3.3653
Epoch 1/4 ___  Discriminator Loss: 0.9158 ___ Generator Loss: 1.1239
Epoch 1/4 ___  Discriminator Loss: 2.1118 ___ Generator Loss: 0.2590
Epoch 1/4 ___  Discriminator Loss: 0.5008 ___ Generator Loss: 3.1269
Epoch 1/4 ___  Discriminator Loss: 0.6338 ___ Generator Loss: 2.7970
Epoch 1/4 ___  Discriminator Loss: 1.6573 ___ Generator Loss: 3.6456
Epoch 1/4 ___  Discriminator Loss: 1.1171 ___ Generator Loss: 0.9078

Epoch 1/4 ___  Discriminator Loss: 1.0571 ___ Generator Loss: 1.2426
Epoch 1/4 ___  Discriminator Loss: 0.8082 ___ Generator Loss: 1.6202
Epoch 1/4 ___  Discriminator Loss: 1.6649 ___ Generator Loss: 0.4220
Epoch 1/4 ___  Discriminator Loss: 0.6111 ___ Generator Loss: 2.0778
Epoch 1/4 ___  Discriminator Loss: 0.6970 ___ Generator Loss: 2.2897
Epoch 1/4 ___  Discriminator Loss: 1.3583 ___ Generator Loss: 3.4745
Epoch 1/4 ___  Discriminator Loss: 0.7660 ___ Generator Loss: 1.5907
Epoch 1/4 ___  Discriminator Loss: 0.7510 ___ Generator Loss: 2.3196
Epoch 1/4 ___  Discriminator Loss: 1.0112 ___ Generator Loss: 1.3331
Epoch 1/4 ___  Discriminator Loss: 0.8357 ___ Generator Loss: 1.8395

Epoch 1/4 ___  Discriminator Loss: 1.4646 ___ Generator Loss: 3.0115
Epoch 1/4 ___  Discriminator Loss: 0.6495 ___ Generator Loss: 1.7516
Epoch 1/4 ___  Discriminator Loss: 1.2253 ___ Generator Loss: 3.9760
Epoch 1/4 ___  Discriminator Loss: 0.6036 ___ Generator Loss: 2.3114
Epoch 1/4 ___  Discriminator Loss: 0.8142 ___ Generator Loss: 2.0179
Epoch 1/4 ___  Discriminator Loss: 1.3385 ___ Generator Loss: 4.1460
Epoch 1/4 ___  Discriminator Loss: 1.4860 ___ Generator Loss: 0.5040
Epoch 1/4 ___  Discriminator Loss: 0.8843 ___ Generator Loss: 1.4256
Epoch 1/4 ___  Discriminator Loss: 0.9501 ___ Generator Loss: 2.5159
Epoch 1/4 ___  Discriminator Loss: 0.9383 ___ Generator Loss: 1.1825

Submitting This Project¶

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_face_generation.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "helper.py" and "problem_unittests.py" files in your submission.